Annealing Chaotic Pattern Search Learning Method for Multi- layer Neural Networks

As a novel optimization technique, chaos has gained much attention and some applications during the past decade. For a given energy or cost function, by following chaotic ergodic orbits, a chaotic dynamic system may eventually reach the global optimum or its good approximation with high probability. To enhance the performance of the pattern search method (PS), which is a derivative-free direct search algorithm, hybrid pattern search method is proposed by incorporating chaos. Furthermore, an annealing strategy is also utilized to eliminate the fluctuation of the chaos in the latter phrase of the process. We test this algorithm on several benchmark problems, such as exclusive-or (XOR) problem, parity problem and Arabic numerals recognition. Simulation results show that the systems can be trained efficiently by our method for all problems.